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Abstract 



, This paper proposes a new algorithm for MIMO cognitive radio Secondary Users (SU) to learn the 

null space of the interference channel to the Primary User (PU) without burdening the PU with any 
knowledge or explicit cooperation with the SU. The knowledge of this null space enables the SU to 
transmit in the same band simultaneously with the PU by utilizing separate spatial dimensions than the 
PU. Specifically, the SU transmits in the null space of the interference channel to the PU. We present a 
q ■ new algorithm, called the One-Bit Null Space Learning Algorithm (OBNSLA), in which the SU learns 

the PU's null space by observing a binary function that indicates whether the interference it inflicts on 
the PU has increased or decreased in comparison to the SU's previous transmitted signal. This function is 
, obtained by listening to the PU transmitted signal or control channel and extracting information from it 

about whether the PU's Signal to Interference plus Noise power Ratio (SINR) has increased or decreased. 
The SU iteratively modifies its null space estimate and the corresponding interference it inflicts on the 



PU and measures the effect this modification has on the PU's SINR in order to refine its null space 
estimate. 

In addition to introducing the OBNSLA, this paper provides a thorough convergence analysis of 
this algorithm. The OBNSLA is shown to have a linear convergence rate and an asymptotic quadratic 
convergence rate. Finally, we derive bounds on the interference that the SU inflicts on the PU as a function 
of a parameter determined by the SU. This lets the SU control the maximum level of interference, which 
enables it to protect the PU completely blindly with minimum complexity. The asymptotic analysis and 
the derived bounds also apply to the recently proposed Blind Null Space Learning Algorithm. 
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DTRA under grant HDTRA1-08-1-0010. The authors are with the Dept. of Electrical Engineering, Stanford University, Stanford 
CA, 940305. 



DRAFT 



2 



I. Introduction 

The emergence of Multiple Input Multiple Output (MIMO) communication opens new directions in 
Cognitive Radio (CR) networks ||l|-|3|] In particular, in underlay CR networks, MIMO technology enables 
the Secondary User (SU) to transmit a significant amount of power simultaneously in the same band 
as the Primary User (PU) without interfering with it, if the SU utilizes separate spatial dimensions than 
the PU. This spatial separation requires that the interference channel from the SU to the PU be known 
to the SU. Thus, acquiring this knowledge, or operating without it, is a major topic of active research, 
as discussed in We consider MIMO primary and secondary systems defined as follows: we assume 
a flat-fading MIMO channel with one PU and one SU, as depicted in Fig. Q] Let H ps be the channel 
matrix between the SU's transmitter and the PU's receiver, hereafter referred to as the SU-Tx and PU- 
Rx, respectively. In the underlay CR paradigm, SUs are constrained not to inflict "harmful" interference 
on the PU-Rx. This can be achieved if the SU restricts its signal to he within the null space of H ps ; 
however, this is only possible if the SU knows H sp . Yi (51], Chen et. al. and Gao et. al. 17|] proposed 
blind solutions to this problem where the SU learns the channel matrix based on channel reciprocity: 
specifically, where the SU listens to the PU's transmitted signal and estimates H ps 's null space from the 
signal's second order statistics. Since these works require channel reciprocity, they are restricted to PUs 
that use Time Division Duplexing (TDD). 

Unless there is channel reciprocity, obtaining H ps by the SU requires cooperation with the PU in the 
estimation phase; e.g. where the SU transmits a training sequence, from which the PU estimates H ps 
and feeds it back to the SU. Cooperation of this nature increases the system complexity overhead, since 
it requires a handshake between both systems and, in addition, the PU needs to be synchronized with 
the SU's training sequence. In [Q], we proposed the Blind Null Space Learning Algorithm (BNSLA), 
which enables a MIMO underlay CR to learn the null space of H ps without learning the full matrix. 
Furthermore, during this learning, the PU does not cooperate at all with the SU and operates as though 
there were no other systems in the medium (the way current PUs operate today). However, the BNSLA 
requires the following observation constraint: at each short training interval, the SU must observe some 
monotone continuous function of the PU's Signal to Noise plus Interference power Ratio (SINR). For 
example, if the PU is using continuous power control, the PU's signal power is a monotone function of 
its SINR. 

This paper makes two contributions. The first contribution is a new algorithm, called the One-Bit Null 
Space Learning Algorithm (OBNSLA), which requires much less information than the BNSLA; namely, 
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Fig. 1. Our cognitive radio scheme. H ps is unknown to the secondary transmitter and v p (t) is a stationary noise (which 



may include stationary interference). The interference from the SU, H ps x s (i), is treated as noise; i.e.. 
cancellation. 



there is no interference 



the SU can infer whether the interference it inflicts on the PU has increased or decreased compared to 
a previous time interval with a one-bit function. In other words, in the OBNSLA the SU measures a 
one-bit function of the PU's SINR, rather than a continuous-valued function as in the BNSLA. Using this 
single bit of information, the SU learns H ps 's null space by iteratively modifying the spatial orientation 
of its transmitted signal and measuring the effect of this modification on the PU's SINR. The second 
contribution of the paper is to provide a thorough convergence analysis of the OBNSLA. We show that 
the algorithm converges linearly and has an asymptotically quadratic convergence rate. In addition, we 
derive upper bounds on the interference that the SU inflicts on the PU; these results enable the SU 
control the interference to the PU without any cooperation on its part. Furthermore, all the bounds and 
the convergence results apply equally to the BNSLA. 

The remainder of this paper is organized as follows: Section [TT] provides the system description 
and notation. Section [III] presents the One-Bit Null Space Learning Algorithm. Section |IV] provides 
a convergence analysis of the algorithm and presents bounds on the interference that the SU inflicts on 
the PU. Simulations and conclusions are presented in Sections |VI] and |VII[ respectively. 

II. The One-Bit Null Space Learning Problem 

Consider a flat fading MIMO interference channel with a single PU and a single SU without interference 
cancellation; i.e., each system treats the other system's signal as noise. The PU's received signal is given 
by 

y p (t) = H P pXp(t) + H ps x s (t) + Vp(t), t e N (1) 

where x p , x s is the PU's and SU's transmitted signal, respectively, H pp is the PU's direct channel; H ps 
is the interference channel between the PU Rx and the SU Tx, and v p (t) is a zero mean stationary noise. 
In the underlay CR paradigm, the SU is constrained not to exceed a maximum interference level at the 
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Fig. 2. The time indexing used in this paper, t indexes the basic time unit (pulse time) where N time units constitute a TC 
that is indexed by n. Furthermore, K transmission cycles constitute a learning phase (not shown in this figure). 



PU Rx; i.e., 



|Hp S x s 



(2) 



where r] max > is the maximum interference constraint. If ry max = 0, the SU is strictly constrained to 
transmit only within the null space of the matrix H ps . In this paper, all vectors are column vectors. Let 
A be an I x m complex matrix; then, its null space is denned as M (A) ={y £ C m : Ay = 0} where 

o = [o,...,o] T eC'. 

Since our focus is on constraining the interference caused by the SU to the PU, we only consider the 
term H ps x s (t) in £[]). Hence, H ps and x s will be denoted by H and x, respectively. We also define the 
Hermitian matrix G as 

G = H*H (3) 

The time line N is divided into iV-lengfh intervals, each referred to as a transmission cycle (TC), as 
depicted in Figure [2] For each TC, the SU's signal is constant; i.e., 



x s ((n - l)N + N') = x s ((n - l)N + 1) 



x 2 (Nn + N' - 1) = x(n), 



(4) 



where the time interval nN < t < nN + N' — 1 is the snapshot in which the SU measures a function 
q(n) that satisfies the following observation constraint. 

Observation Constraint (OC) on the function q(n): There exists some integer M > 1, such that 
given q(n — m), q(n), the SU can extract 



/i(x(n), x(n — m)) 



1, if ||Hx(n)|| > ||Hx(n-m) 
— 1, otherwise 



(5) 



for every m < M. 

The SU's objective is to learn A/"(H) from {x(n), q(n)} n& ^. This problem, referred to as the One-bit 
Blind Null Space Learning (OBNSL) problem, is illustrated in Figure|3]for M = 1. The OBNSL problem 
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h(x(n),x(n — 1)) 



\m(n)\\i ||Hx(n-l)|| 



Fig. 3. Block Diagram of the One-bit Blind Null Space Learning Problem. The SU's objective is to learn the null space of H 
by inserting a series of {x(n)} nf =n and measuring /i(x(?i), x(n — 1)) as output. 



is similar to the Blind Null Space Learning (BNSL) problem I4J] except for one important difference. 
In the latter, the SU observes a continuous-valued function of the PU's SINR whereas in the OBNSL 
problem, it observes a one-bit function. In both problems, the SU obtains q(n) by measuring the PU's 
transmit energy, or any other parameter that indicates the PU's SINR (see Sec. II-B in [4] for examples). 
However, in the OBNSL problem, the SU is more flexible since it can obtain q{n) from, for example, 
incremental power controU or other quantized functions of the PU's SINR such as modulation size. From 
a system point of view, the OC means that between m consecutive transmission cycles, the PU's SINR 
is only affected by variations in the SU's signal. The learning process unfolds as follows. In the first TC 
(n = 1), the SU transmits x(l), and measures q(n). In the next TC, the SU transmits x(2) and measures 
q{2) from which it extracts /j(x(1), x(2)). This process is repeated until the null space is approximated. 
Note that while /i(x(l), x(2)) requires two TCs, /i(x(n — l),x(n)) for n > 2 requires a single TC. In 
the following section we describe the algorithm that performs this learning. We will also discuss the role 
of M in the OC. This parameter, which indicates the "memory" that q(n) has of the PU's SINR, may 
affect the number of TCs required for learning A/"(H); i.e., a larger M reduces the number of TCs. 

III. The One-Bit Blind Null Space Learning Algorithm (OBNSLA) 

We now present the OBNSLA by which the SU approximates AA(H) from {x(n), q(n)}^ =1 under 
the OC, where the approximation error can be made arbitrarily small for sufficiently large T. Once the 
SU learns jV(H), it can optimize its transmitted signal, regardless of the optimization criterion, under 
the constraint that its signal lies in A/"(H). Let USV* be H's Singular Value Decomposition (SVD), 
where V and U are n t x n t and n r x n r unitary matrices, respectively, and assume that nt > n r . The 
matrix £ is an n r x n t diagonal matrix with real nonnegative diagonal entries a± , . . . , cr^ arranged as 

'This is power control that is carried out using one-bit command which indicates weather to increase or decrease the power 
by a certain amount. 
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01 > o"2, > ■ ■ ■ > ad > 0. We assume without loss of generality that n r = d(= Rank(H)). In this case 

Af(H) = span(v nr+ i, v n J , (6) 

where Vj denotes V's ith column. From the SU's point of view, it is sufficient to learn A/"(G) (recall, 
G = H*H), which is equal to jV(H) since 

G = VAV*, (7) 

where A = £ T E. The decomposition in (O is known as the Eigenvalue Decomposition (EVD) of G. 
In order to obtain JV(H) it is sufficient to obtain G's EVD. However, in the OBNSL problem, G is not 
observed, so the SU needs to obtain the EVD using only one-bit information. The OBNSL algorithm 
does so by blindly implementing the well-known Cyclic Jacobi Technique (CJT) for Hermitian matrix 
diagonalization. We begin with a review of that technique. 

A. Review of the Cyclic Jacobi Technique 

The Jacobi technique [see e.g. Q] obtains the EVD of the n t x n t Hermitian matrix G via a series of 
2-dimensional rotations that eliminates two off-diagonal elements at each step (indexed by k). It begins 
by setting Ao = G and then performs the following rotation operations A^+i = VfeA^V^, k = 1,2, 
where 

V k = R lim (9,4>) (8) 

is an n t x n t unitary rotation matrix that is equal to I„ f except for its mth and Zth diagonal entries that 
are equal to cos(#), and its (m,/)th and (Z,m)th entries that are equal to e~^sm(6>) and — e 1 ^ sin(0), 
respectively. For each k, the values of 6,(p are chosen such that [Af.]i m = 0, or stated differently, 6 
and <$> are chosen to zero the l,m and m,l off diagonal entries of A& (which are conjugate to each 
other). Note that in an nt x n t Hermitian matrix, there are {n t — l)n^/2 such pairs. The values of l,m 
are chosen in step k according to a function J : N — > {1, ...,n t } x {1, ...,n t }, i.e = (Zfe,mfe). It is 
the choice of that differs between different Jacobi techniques. In the cyclic Jacobi technique, Ik, ink 
satisfy 1 < If. < n t — 1 and < m\~ < n t such that each pair (Z, m) is chosen once in every (n t — l)nt/2 
rotations. Such (n t — l)nt/2 rotations are referred to as a Jacobi sweep. An example of a single sweep 
of the CJT for n t = 3 is the following series of rotations: J\ = (1, 2), J2 = (1, 3), J3 = (2, 3). The next 
sweep is J4 = (1, 2), J5 = (1, 3), J§ = (2, 3) and so forth. 

The convergence of the CJT has been studied extensively over the last sixty years. The first proof of 



DRAFT 



convergence of the CJT for complex Hermitian matrices was given by Foster and Henrici [9fl. However, 
this result did not determine the convergence rate. The convergence rate problem was addressed by 
Henrici and Zimmermann 111 Oil , who proved that the CJT for real symmetric matrices has a global 
linear convergence rate^l that depends on the matrix size n t if 9^ G [— 7r/4, tt/4] for every k. Fernando 
Jlli extended this result to complex Hermitian matrices. It was later shown by Henrici [12], and by 
Wilkinson I13I that in the case of a complex Hermitian matrix with well separated eigenvalues, the CJT 
has a quadratic convergence^ rate. Hari [14j] extended the results of Henrici and Wilkinson by proving a 
quadratic convergence rate under more general conditions, including identical eigenvalues and clusters of 
eigenvalues (that is, very close eigenvalues). Studies have shown that in practice the number of iterations 
that is required for the CJT to reach its asymptotic quadratic convergence rate is a small number, but 



this has not been proven rigorously. Brent and Luk 111511 have argued heuristically that this number is 



0(log 2 (n i )) cycles for n t x n t matrices. Extensive numerical results show that quadratic convergence 
is obtained after three to four cycles (see e.g. [8, page 429], llrl page 197]). Thus, since each Jacobi 
sweep has n t {n t — l)/2 rotations, the overall number of rotations in the CJT rou ghly grows as rtf. For 



further details about the CJT and its convergence, the reader is referred to Q8 , 



16 



13. 



B. The One-Bit Line Search 

The learning in the OBNSLA is carried out in learning stages, indexed by k, where each stage 
performers one Jacobi rotation. The SU approximates the matrix V by Wfc 3 , where 

W k = Wfc^R^^fc, <f> k ), k = 1, k a , (9) 

and Wo = I. Recall that in the Cyclic Jacobi technique, one observes the matrix 

A fc -i = WjUGWfc-! (10) 
and chooses the rotation angles which zero A^'s (Z, m)th off diagonal entry; i.e., 



[R;, m (^,^)A w R iim (^^)] iim = (11) 

2 A sequence a n is said to have a linear convergence rate of < /3 < 1 if there exists no G N such that |on+i| < fi\a n \ for 
every n > no. If no = 1, «« has a global linear convergence rate. 

3 A sequence is said to have a quadratic convergence rate if there exists /3 > 0, no £ N such that |o n +i| < /3|a n | 2 , Vn > no. 
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In the OBNSL problem, the SU needs to perform this step using only {x(n), q(n)} n= i, without observing 
the matrix Ak-i- The following theorem, proved in |4j], is the first step towards such a blind implemen- 
tation of the Jacobi technique. The theorem converts the problem of obtaining the optimal Jacobi rotation 
angles into two one-dimensional optimizations of the function S^A^— 1, 17 m (9, 4>)) (which is continuous, 
as shown in [4J]), where 5(A,x) = x*Ax and ri m (9,<fi) is R; m (0,0)'s Zth column. 

Theorem 1: [|4j, Theorem 2] Consider a.n t xn t Hermitian matrix A&_i in (fTOl) . and let S( A, x) = x* Ax 
and ri im (6, (p) be R; )Tn ,(0, 0)'s Zth column. The optimal Jacobi parameters 9j, and cf)^, which zero out the 
(/,m)th entry of R* m (0fc, ^)A fe _ 1 R^ m (^, are given by 



argmin5(A fc _i,r ijm (7r/4, <f))) (12) 

0G[— 7T,7r] 

T k (4) (13) 



where 



TM = < W) " - 5 S *<« £ * (14) 

9fe(</>) — sign(0fc((/)))7r/2 otherwise 

where sign(x) = 1 if a; > and —1 otherwise, and 

£ fc (0)=arg min S 1 (A fc _i, r i)m (0, 0)) (15) 

6»e[-7r/2,7r/2] 

The theorem enables the SU to solve the optimization problems in (ITZb and (031 ) via line searches 
based on {x(n), g(n)}^ =1 . This is because under the OC, the SU can extract h(x(n), x(n — m)), which 
indicates whether 5(G, x(n)) > S*(G, x(n — m)) is true or false. It is possible, however, to further reduce 
the complexity of the line search, which is important, since each search point requires a TC. To see this, 
consider the line search in (fT2l and denote w(<p) = S(Af.,ri m (7r/4,(/))) = ||HW/-_ir; m (7r/4, </>)|| 2 . 
According to the OC, for each </>i,</>2 the SU only knows whether w{(f>\) > w(4>2) or not. Assume that 
the SU tries to approximate ^ by searching over a linear grid, with a spacing of 77, on the interval [—it, it]. 
The complexity of such a search is at least 0(1/ rj) since each point in the grid must be compared to 
a different point at least once. The two line searches in (PT21 and (fl~5l ) would be carried out much more 
efficiently if binary searches could be invoked. However, a binary search is feasible only if the objective 
function has a unique local minimum point, which is not the case in (fl"2l and ( fT5l ) because 

S(G,r, m (e, 4>)) = cos 2 (0) |s» ,| + sin 2 (0) | 

9m,m\ \9l,m 

I sin(2#) cos(0 + Zm m ) (16) 
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Thus, before invoking the binary search, a single-minimum interval (SMI) must be determined; i.e, an 
interval in which the target function in ([TZt or <fT5T > has a single local minimum. This is possible via the 
following proposition: 

Proposition 2: Let w{4>) = ||Hrj jm (7r/4, (f))\\ 2 where rj m is defined in Theorem [TJ Let cj) £ [— t^tt] 
be a minimurro point of w (</>), then 

(a) 4> G [-3tt/4, -tt/4] if w(-tt), w(0) > w(-tt/2). 

(b) 4> G [-7r/4,7r/4] if w(-vr) > w(-tt/2) > w(0). 

(c) 4> G [tt/4, 3tt/4] if w(-ir), w(0) < w(-n/2). 

(d) 4> G [3vr/4, tt] U [-7T, -3vr/4] if w(-vr) < io(-tt/2) < w(0). 

Proof: From ( fT6l ), w(</>) can be expressed as w(4>) = B — A cos(cj) — <p), A, B > 0. If B = 0, every 
G [— 7r, 7r] is a minimum point and the proposition is true. We now assume that B > 0. By substituting 
^(0) = B-Acos(4>),w{'k/2) = B-Asm((f>) and w(ir) = B + Acos(cp) into w(-n), w(Q) > w(-n/2), 
one obtains that the latter is equivalent to ±cos(0) > sin(^) which is equivalent to (— 37r/4 < < 
7r/4) n ((-tt < (f> < -7r/4) U (37t/4 < < 7r)). The last set can be written as G (— 37r/4, — 7r/4), 
which establishes ([a]). The proof of (b)-(d) is similar. □ 
Note that unless w is a horizontal line, it is a 2tt periodical sinusoid. In the latter case, there cannot be 
more than a single local (and therefor global) minimum within an interval of tt/2. If w is a horizontal 
line, every point is a minimum point. In both cases, a binary search can be invoked; i.e., the SU can 
efficiently approximate 0l,(f>l by Q\ and (jr k , respectively, using a binary search, such that 

\9i-TSl)\,\4>l-4\<r ] , (17) 

where rj > determines the approximation accuracy. In order to invoke a binary search, the SU uses 
Proposition |2 to determine an SMI via n n (7r, tt/2) and n n (7r/2, 0), where u n ((j) n , 4> n -i) = h n (ri )m (iT /4, 4> n ), 
r; m (7r/4, <ft n _i)), and h is defined in (f5]). The one-bit line search is given in AlgorifhmQ] In determining 
the SMI, the one-bit line search requires 3 TCs: two TCs for u n (7r,7r/2), and one more for u n (7r,0). 
Given an SMI of length a and an accuracy of r] > 0, it takes [— log 2 (?//a)J + 1 search points to obtain 
the minimum to within that accuracy. In the search for (ftj,, a = tt/2, thus (f>l G — rj, <f>^ + i]] is 
obtained using [— log(277/-7r)] + 1 TCs, plus the 3 TCs required for determining the SMI and 2 more TCs 
to compare the initial boundaries of the SMI. Then, 6^, can be approximated to within rj in the same 
way. 

4 Since w is a 2tt periodical sinusoid, such a point always exists, though might not be unique. 
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We conclude with a discussion of parameter M in the OC. The proposed line search can obtain w(fa's 
minimum even for M = 1. However, the number of TCs is lower if M is larger. Assume that the SU 
has obtained an SMI [(/> min , max ]. It takes the SU two TCs where, in the first, it transmits r/ m (7r/4, fa), 
where fa = fa n - m and measures q(l), and in the second TC, it transmits j-j m (7r/4, fa), where fa = (j) max 
and measures q(2). These q's are sufficient for determining whether 

^Omax) > w((j) min ) (18) 

If ( fT8l ) is false, max is set as fa lSLK = (fa n - m + <?Wx)/2. In the next phase of the binary search the 
SU transmits j-j m (7r/4, ^3), where fa = 4> max and measures q(3). However, because M = 1 it cannot 
use q(l) to check ( fl~8l ); it needs an additional TC to do so. This extra TC is not required if M > 1. In 
general, M > [— \og(r]/ir) + lj (which is the maximum number of search points required for the binary 
search) guarantees that each search point requires one TC. 

C. The OBNSLA 

Now that we have established the one-bit line search, we can present the OBNSLA. In the OBNSLA, 
the SU performs two line searches for each k. The first search is carried out to find fa that minimizes 
||HWfcr^ mfc (7r/4, </>))|| 2 , where each search point, fa, is obtained by one TC in which the SU transmits 

x s (t) = x(n) = W fc r /fc , mfc (tt/4, fa) E O , V(n - 1)N < t < nN, (19) 

and measures q(n). In first line search, the SU obtains fa which is then used in the second line search 
to obtain 0k{fa) according to (fl~5T l. and then to obtain Oj. according to ([T"3T l. The indices (Zfe,mfe) are 
chosen as in the CJT 

After performing k s iterations the SU approximates the matrix V (see (|7])) by Wfc s . It then chooses 
its pre-coding matrix Tfc s as 

T jb . = [wt,...,w^_J, (20) 
where is W^'s ith column, and 21,^2, *n t i s an indexing such that 

for every q < v. Thus, the interference power that the SU inflicts on the PU is bounded as ||Hx|| 2 < 
p s ||Hw^ s || 2 ,V||x|| 2 = p s , where p s is the SU's transmit power. The OBNSLA algorithm is given 
in Algorithm [2] It is important to note that since only A n (x(n),x(n — 1)) is observed, rather than 
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|Hx(n — 1)|| 2 and ||Hx(n)|| 2 , the eigenvalues of G cannot be obtained by the OBNSLA^. 

Although the SU becomes "invisible" to the PU after it learns J\f(H ps ), it interferes with the PU during 
this learning process. Furthermore, this interference is an important ingredient in the learning since it 
provides the SU with the means to learn J\f(H ps ), i.e. q(n). Nevertheless, the SU must also protect the 
PU during the learning process. Hence we assume that there exists an additional mechanism enabling 
the SU to choose x(n)'s power to be high enough to be able to extract q(n), but not too high, so as to 
meet the interference constraint (|2). We give examples for such mechanisms in [4], Sec. II-C]. 

Algorithm 1 [z,n] = OneBitLineSearch({/iz}j e N,£ max ,n, 7 ?j x ( 2; )) 
Initialize: L <— z max , 
u n (zi,z 2 ) <- h n (x(zi),x(z 2 )) 
a «- u n (-L, -L/2),n + +. 
b<-Un(0,-L/2), n + + 
z mH <- (3 + 2b - 2o(l + 26))L/4; 

^min ^ Zmax L/2. 

while |z max - z min \ > j] do 

Z 4 (-Zmax ~r~ Zmxn)/2 

a 4r- ti(z max , Zmin), fl + + 

if a=l then 

•Zmax ^ Z 

else 

•Zmin ^ Z 

end if 
end while 



IV. Algorithm Convergence 

The OBNSLA is, in fact, a blind implementation of the CJT whose convergence properties have been 
extensively studied over the last 60 years. However, the convergence results of the CJT do not apply 
directly to the OBNSLA. This is because of the approximation in (fTTT ); i.e., due to the fact that for every 
k, the rotation angles 6^, are obtained by a binary search of accuracy rj. Thus the off diagonal entries 
are not completely annihilated; i.e., [A&+i]j fc mk ~ instead of [Afc +1 ]^ , mfc = 0. Moreover, we would 
like to make this line search accuracy as low as possible (that is, to make r\ as large as possible) in 
order to reduce the number of TCs. It is therefore crucial to understand how rj affects the performance 
of the OBNSLA algorithm, in terms of convergence rate and the interference reduction to the PU. In this 

5 In ITHl it is shown that if S(G, x) is known, the problem can be simplified drastically where G can be obtained precisely 
by a finite number of TCs. 
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Algorithm 2 The OBNSL Algorithm 

Input: {h v } V £$i, defined in ([5]). 

Output:W 

initialize: n = 1 

[W,n] = OBNSLF({^} u6N ,n t ,n) 
End 

Function: [W,ra]= OBNSLF({h„}„ e N, nt, n) 

Initialize: k = 1, W = l nt , Aj = 2r/,\/j < 
while (max J - G{fc _ nt(nt _ 1 ) /2) ... )fc} Aj > ? ? ) do 
x(0)^-Wr Ifc|mft (7r/4,0) 

[(^fc,n] OneBitLineSearch ^{/fy}^, 7r, ra, ?y, x(c/>)^ 
x(0)<-Wr Ifcjmh (0,^) 

n] <— OneBitLineSearch ^{/i/}; e N, tt/2, n, n, x(6*)^ 
4 <- 6 k if fc < |vr/4|, otherwise 6 k <- fc - 7rsign(6i fc )/2. 
A fc <- |flfc| 

W^W%, mt (^,#) 
fe«- fc + 1. 
end while 



section, we extend the classic convergence results of the CJT to the OBNSLA and indicate the required 
accuracy in the binary search that assures convergence and bounds the maximum reduction level of the 
interference inflicted by the SU on the PU. It will also be shown that the same convergence analysis 
applies to the BNSLA proposed in Q4[| . 

A. Global Linear Convergence Rate 

The following theorem shows that for a sufficiently good line search accuracy, the OBNSLA has a 
global linear convergence rate. 

Theorem 3: Let G be a finite dimensional n t x n t complex Hermitian matrix and P k denotes the 
Frobenius norm of the off diagonal upper triangular (or lower triangular) part of A k = WLjGW^i 
where is defined in (O and let m = nt(rit — l)/2. Let r\ be the accuracy of the binary search (see 
(fTTl)). then the OBNSLA satisfies 

P 2 k+m < Pi (1 - 2-K" 2 )^- 1 )/ 2 ) + (n? - n t )(7 + 2V2) V 2 \\G\\ 2 (21) 
Proof: See Appendix [A] 



Comment: For 77 = 0, we obtain the well known convergence result by Fernando 111 ill . 
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B. An Asymptotic Quadratic Convergence Rate 

So far, it has been shown that for the right choice of rj, the OBNSLA converges and that for sufficiently 
small rj it has an approximately global linear convergence rate. In what follows, it will be shown that for 
sufficiently small r], the OBNSLA has an asymptotic quadratic convergence rate, but in order to obtain 
this, we modify the algorithm slightly as follows. Let / : {1, n t } — > {1, n t } be the identity operator, 
i.e. I(x) = x. At the beginning of each sweep, i.e. for every k = q(n^ — nt)/2 were g£N, the SU sets 
I q = I and for each k E {q{nf — n t )/2 + 1, (q + l)(nf — n t )/2}, the SU modifies I q as follows 

, , I h ^ a l k l k > a m k ,m k 

m& otherwise 

At the end of each sweep, i.e. for k = (q + l)(nf — n t )/2, the SU permutes the columns of 
such that Wfc's Ith column becomes its J ? (/)'s column. Note that this modification does not require 
extra transmission cycles since all the additional calculations are carried out at the SU's transmitter. 
Furthermore, the convergence result in Theorem |3] is still valid. We refer to the OBNSLA after this 
modification as the modified OBNSLA. 

Besides the fact that this modification is necessary for guaranteeing the quadratic convergence rate, as 
will be shown in the following theorem, it will also be shown that it helps the SU to identify the null 
space (the last n r columns of W&) blindly without taking extra measurements. 

Theorem 4: Let r\ be the accuracy of the binary search, {A;}[!l 1 be G's eigenvalues and let 

5 = min |A; — AJ/3 (23) 

Let Pk be the Frobenius norm of the off diagonal upper triangular part of = W^jGWjt-i, where 
Wfc is defined in (O and let m = (raf — n t )/2. Assume that the modified O/BNSLA has reached a stage 
k, such that P| < 8 2 /8, then 

2 ^nff^\ 2 \^nf^ 



C<0 t +0 +0 ^ +2(n?-n,)^||Gf (24) 



Furthermore, the last nt—n r columns of Wfe +m inflict minimum interference to the PU; i.e., HHp^-w^" 1 "" 1 !! < 
||Hp S wf +m ||, VI < j < n r < i < n t . 
Proof: See Appendix 151 

Theorem [4] shows that to guarantee the quadratic convergence rate, the accuracy, r\, should be much 
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smaller than P%; that is, let ko be an integer such that P^ g < 5 2 /8, then 

^ +2m <o((^-) 2 ) (25) 

if rj « P£ . This implies that once P^ becomes very small such that P k = 0{rj), one cannot guarantee 
that Pk+2m will be smaller than P k+m since at k + 1 it will be 0(rj). 

The asymptomatic quadratic convergence rate of Theorem @] is determined by 1/5 where 35 is the 
minimal gap between G's eigenvalues. In addition, the quadratic convergence rate takes effect only after 
P 2 < 5/8. Such a condition implies that if 5 is very small, it will take the modified OBNSLA many cycles 
to reach its quadratic convergence rate. This is problematic since MIMO wireless channels may have very 
close singular values (recall that Hi2's square singular values are equal to G's first n r eigenvalues). If 
we were using the optimal Cyclic Jacobi technique (i.e. no errors because of finite line search accuracy) 
this would not have practical implications since a quadratic decrease in Pk, which is independent of 5, 

9 I I 

occurs prior to the phase where P£ < 5/8 1X411 . In the following theorem, we extend this result to the 
modified OBNSLA. 

Theorem 5: Let rj be the accuracy of the line search, {A^}^ be G's eigenvalues such that there 
exists a cluster of eigenvalues; i.e., there exists a subset {Aj,}T =1 C {A/}™* such that Aj ( = A + £j, for 
I € L2 = {ii, i v }, where Ya=i 6 = and the rest of the non-equal eigenvalues satisfy 5 C > 16</^^ 2 , 
where 

35 c = min(Ai U A 2 ) (26) 
Ai = {|A,-A r | :l€L\L 2 , Xi^Xr} 

A 2 = {|A ; -A| :l€L\L 2 } (27) 
L = {1,... ,n t } 

Then, once the modified OBNSLA reaches a k such that 

J^Jf<Pi<S 2 c /8 (28) 

it satisfies 

Pl +2m < O ((%-) 4 ) + O ((%-)) + O ((^)) +2(nf- n t ) ^\\Gf (29) 
Proof: See Appendix ICl 

Theorem [5] states that in the presence of a single eigenvalue cluster; i.e. yYli^f ^c> and if 
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rjk = o(Pk), the modified O/BNSLA has four convergence regions: The first region is Pj: > <5 2 /8, 

2 E>2 ^ x2 /o t u Q tu;^A -,c X /Q ^ d2 ^ ox /V"* e2 



the second is 25^^^ < P fe 2 < 6~/8, the third is 5/8 < P fc 2 < 2<W^£f and the fourth is 
P| < min; £ 2 /8. In the first and the third regions, the modified OBNSLA has at least a linear convergence 
rate while in the second and fourth regions, it has a quadratic convergence rate. This means that from a 



practical point of view, a close cluster of eigenvalues; i.e. y X^£ 2 /5 C << 1, is not a problem. This is 
because once the algorithm enters the second convergence region; i.e., it reaches the stage k = &2 such 



that 25 c yJY,i %i « P k 2 ^ <W 8 > p k will decrease quadratically until k = k 3 such that P fe 2 < 2<W^ £r 
But the latter inequality implies that P^, 3 << P^ 2 , a fact that guarantees a significant reduction P^; i.e., 
from i\ to Pfc 3 with a quadratic rate. 

Nevertheless, Pk will eventually decrease quadratically as P 2 becomes smaller than 5/8 as required 



by Theorem |U This phenomenon is also a characteristic of the Cyclic Jacobi technique IU4I1 . 



C. The Asymptotic Level of the Interference to the PU 

In the previous theorems, we discussed the convergence rate of the sequence P&; i.e., the root sum of 
squares of A^'s off diagonal entries. We now consider the maximum level of interference that the SU 
inflicts on the PU. Our aim here is to relate the asymptotic behavior of the maximum interference to 
that of Pfc, and to obtain bounds on the maximum interference as a function of r\. We begin with the 
following proposition: 

Proposition 6: Let be the SU's pre-coding matrix defined in (l20l i. t\ be its ith column, Q = 
{l,...,n t — n r }, and P& be the norm of the off diagonal upper triangular (or lower triangular) part of 
Afc (where is defined in (fTOb). Then 

max||Hi2^|| 2 < 2Pl (30) 



Proof: This is an immediate result of 11191 Corollary 6.3.4] which states that for every eigenvalue A 
of B + E, where B is an n t x n t Hermitian matrix with eigenvalues Aj, i = 1, n t , there exists Aj such 
that |A — Aj| 2 < ||E|| 2 , where || • || is the Forbinus norm. Thus, if one expresses A^ as A& = B + E, 
where B = diag(A fc ),E = offdiag(A fc ), ([30]> follows. □ 
Since the maximum interference to the PU is bounded by 2P 2 (from Proposition [6]), it is possible 
to apply the results of Theorems @] and [5] and to bound the maximum interference. These bounds are 
valuable since they relate the asymptotic level of interference to the accuracy of the line search 77 (which 
is determined by the SU), thus enabling the SU to control the interference reduction to the PU. 

Before obtaining the first bound on the interference, we need the following corollary of Theorem [3] 
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Corollary 7: 



^ (n?-nQ(7 + 2v^ 2 ||G|| 2 
hmsupP fc < 2 „ (nt _ 2)(nt _ 1)/2 • (3D 



Proof: See Appendix |Pl 
From Corollaries [6] and [7] we obtain the following bound: 

„ TT kn2 2(n? -n t )(7 + 2^)r ? 2 ||G|| 2 
bmsupnu* ||H 12 *Jf < ' (32) 

We now derive a tighter bound than (|32l l which is valid only if the conditions of Theorem |4] are 
satisfied; i.e., that the OBNSLA is replaced by the modified OBNSLA and that there exists k such that 
Pi < 5 2 /8. In this case, by combining Proposition [6] and Theorem [4] one obtains 

max qeQ \\U 12 t k X < O (f) 2 + O {^P) + O (^) + 2 (n 2 - n t ) V 2 \\G\\ 2 (33) 

Furthermore, if P^ becomes sufficiently small such that n > Pj~, the dominant term in the RHS of d33l ) 
will be 0(r] 2 ); i.e., we effectively have: 

max ||H 12 ^|| 2 < 2 (n 2 - n t ) r, 2 \\G\\ 2 + ©(r/ 2 - 5 ) (34) 

V. The Reduced Complexity Blind Null Space Learning (RC-BNSL) 

In this section we present the Reduced Complexity MBNSL (RC-MBNSL) algorithm. In the BNSL 
algorithm, each step requires two binary searches, where each search point is obtained by a transmission 
cycle. These transmission cycles are the dominant latency factor in the learning process since the rest 
of the calculations are performed offline at the secondary device processing unit. Roughly speaking (as 
discussed in Section ITlI-AI) . the number of rotations (recall that each Jacobi rotation is a learning stage in 
the M/BNSL algorithms) of the Cyclic Jacobi technique, for daigonalizing a Hermitian matrix G grows 
like n 2 , the dimension of the matrix G. In this section it is shown that if G is low rank, its diagonalization 
can be simplified to (n t — n T ) diagonalizations, each one of a single n r x n r matrix. This is possible due 
to the fact that G = H*H is an n r -rank matrix and therefore has an (n t — n r ) -dimensional null spaced 
The resulting number of stages of each Jacobi sweep grows like (n t — n r )n 2 where in the M/BNSL 
it grows like n 2 . Therefore, the proposed RC-BNSL algorithm is more efficient than the BNSLA if nt 
is sufficiently larger than n r , that is, if the SU has more antenna at its transmitter than the PU has in 

6 lf the matrix G was known, the fact that rank(G) = n r could have been utilized by QR decomposition (which cannot be 
done blindly) prior to the diagonalization. Note while the SU does not know G, we assume that he know its rank n r . This is 
possible if the SU know what kind of PU uses the channel. 
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its receiver. This complexity reduction is significant if the SU is a very large MIMO i.e. for wireless 

communication with tens or even hundreds of antenna [? ]. 

The idea behind the RC-BNSL algorithm is described in the following observation: 

Observation 8: Let H G C" rXnt be an n r -rank (n t > n r ) matrix and let U = [ui, u„ r+ i] G 

(£n t x(n r +i) jj e an orfhonormal matrix (that is, a matrix whose columns are an orfhonormal set i.e. U*U = 

I) and let H = HU. If u G W(H) then u = Uu G jV(H). 

Proof: This is due to Hu = HUu = since u G jV(HU). 
The RC-BNSL algorithm is carried out as follows: The secondary user begins with an initial pre- 

coding matrix U^ ) G ^n r x(n t -n r ) w hj c h j s composed of the last n t — n r columns of some unitary 



matrix W G C 



n t xn t 



. Let H 



(i) 

cq 



HU(°) G C n " x( - n '- +1 \ then there exists at least one degree of freedom 



in this channel. The SU can apply the BNSL algorithm on G^ 1 ) = Heq Heq and obtains a pre-coding 



matrix uj^ such that = lim 



fc^oo Ufc and that 



A« = U( 1 ) , G«UW 




A« 



A 





(i) 



(35) 



Now that G^ 1 ) is diagonalized, the first degree of freedom is given by = U^u^ where is the 
first column of (that lies in the null space of G^). The SU then can gain an additional degree of 
freedom by applying the BNSL algorithm on the following (m + 1) equivalent channel 



H$=HU« 



■■cq 



(36) 



where U^ 1 ) G C ntXUr+1 is obtained by concatenating the n r — 1 columns of the initial unitary matrix 
W with the last n r column of multiplied by U^, i.e. let = [u^, ...,u^ +1 ], then U^ 1 ) = 
[w nr _i, U^CrW]. This equivalent channel is then diagonalized using the BNSL algorithm to obtain 
TjW. We now have two degrees of freedom given by v( 2 ) = UWuf } and vW. This process is repeated 
until all W's columns are used. The RC-BNSL algorithm is summarized in Algorithm [3] 

We conclude with the following corollary, which extends the convergence analysis presented in this 
section to the BNSLA. 

Corollary 9: Theorems |3l 01 [51 Proposition |6] and Corollary |7] apply to the BNSLA presented in Q4Q . 
Proof: The proofs of Theorems |3j |U |5j Proposition |6] and Corollary [7] rely on the fact that the 
only difference between the CJT and the OBNSLA is in the rotation angles. In the OBNSLA, the CJT's 
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Algorithm 3 The RCBNSL algorithm 

Input: n t ,n r , {h v } v& ^, defined in ©, W e C ntXn \ s.t. W*W = I 
Initialize: U (0) = [w„ t _ nr , w„J 
for m = 1, . . . , nt — n r do 

a n (x 1 ,x 2 )^^ n (U(" l - 1 )x) 

UM = MBNSL({ fli ,}„ eN ,n r + l,n) 

v m = U^-^u^ 

c m = [fiS m) ,..,fifti] 

U(-) = [wi7l nr _ m ,U(— ^UM] 
end for 



rotation angles , Ol, (jr k , are approximated according to ([P7V Furthermore, note that the BNSLA and the 
OBNSLA are identical except for the way in which each algorithm determines its SMI (which are not 
identical SMIs) before invoking the binary search. However, (TTvT t is satisfied as long as each SMI contains 
the desired minimum point. Because the latter is satisfied by both algorithms, as indicated by Proposition 
12 for the OBNSLA and by Proposition 3 in If for the BNSLA, Theorems E g] |5] Proposition |6] and 
Corollary E] apply to the BNSLA. □ 

VI. SIMULATIONS 
A. N on- Asymptotic Comparison of the BNSLA and the OBNSLA 

In this section we compare the OBNSLA to the BNSLA. In this simulation the PU performs a power 
adaptation every 1 msec to maintain a target 10 dB SINR at the receiver, and the SU inflicts interference 
on the PU and measures q(n) by listening to the PU signal's power at the SU-RxQ. Fig. |4] presents the 
interference reduction of the BNSLA and the OBNSLA as a function of H ps 's Doppler spread. The result 
shows that both algorithms have similar performances. 

An important practical issue in the implementation the OBNSLA and the BNSLA is granularity in 
the PU's SINR; i.e., let SINR(n) be the PU's SINR at the nth TC, and let SINR q = [SINRf n , 
SINR™*), q = l, ...,Q, be the granularity of the PU's SINR, that is, if SINR(n), SINR(n + 1) e 
SINR q , the PU will not modify its transmission scheme. Note that if ||Hx(n)|| 2 < ||Hx(n — 1)|| 2 
and the difference between them is small, such that SINR(n), SINR(n - 1) G SINR q , the SU will 
observe /i(x(n),x(n — 1)) = 1, which falsely indicates that ||Hx(n)|| 2 > ||Hx(n — 1)|| 2 . Such errors 
may be significant since the binary search is directed to the wrong interval. A full theoretical convergence 

7 TheSU setsg(n) = i E^jvrf'" 1 ||y s (t)-y s |l 2 , where y s is the average of y s (t) overt = N(n-1)+N', . ..Nn+N'-l. 
The consideration for choosing such q are described in Sec II-B1]. 



DRAFT 



19 



a 

8 

a -10 









/ 








- -OBNSLA 




- ♦ - BNSLA 



5 10 

Doppler Spread [Hz] 



(a) 



° -10 



i. 



■ e - OBNSLA 

■ ♦ - BNSLA 



6 — -o- 



2 3 4 5 6 
Quantization [bits] 



(b) 



Fig. 4. Interference reduction after a single Jacobi sweep as a function of H ps 's Doppler spread. The maximum power 
constraint of each Tx is 23 dBm. The channels' path-losses are calculated as 128.1 + 37.61og 10 (i?), where R is the distance 
between the Rx and the TX in meters as used by the 3GPP (see page 61 3GPP Technical Report 36.814). The locations of the 
PU-TX is randomly chosen from a uniform distribution over a 300 m disk, and the locations of the SU-Tx and the SU-Rx are 
randomly chosen from a uniform distribution over a 400 m disk. Both disks are centered at the location of the PU-Rx. The 
minimum distance between the PU-Rx to the PU-Tx, and the PU-Rx to the SU-Tx is 20 m and 100 m, respectively. For each t 
the entries of the channel matrices TH pp (t), H sp (t), H sa (t) are i.i.d. where each entry is 15 KHz flat fading Rayleigh channel 
with 15 Hz Doppler spread in. H ps (t) has distribution as the other channel, except for its Doppler spread which is given by 
the horizontal axes of both subfigures. All channels are generated using the Improved Rayleigh Fading Channel Simulator |2Ch . 
The noise level at the receivers is -121 dBm and the SU transmit power during the learning process is 5 dBm. The numbers of 
antennas are nt a = 2 n tp = 2, n Tp = 1 n rs = 2. 



analysis of this problem is an important topic for future research. In this paper, we test this problem 
using a simulation. Fig. |4(b)| presents simulation results for a scenario where the PU's power control 
process is based on a quantized measurement of the PU's SINR in the range —5 dB to 20 dB. It is shown 
that the interference reduction is not improved for more than 4 bit quantization. This means that small 
granularity does have a practical affect on the performance of the O/BNSLA. 

B. Asymptotic analysis 

We now compare the asymptotic properties of the OBNSLA to the bounds derived in Section UVl 
Figure [5] presents simulation results for the OBNSLA under optimal conditions; i.e, q{n) is perfectly 
observed, for different levels of line search accuracies. Figure |5(a)| depicts Pk and the bound on it as 
given in (l32l ). versus complete OBNSLA sweeps; i.e. (nf — n t )/2 learning phases. It shows that for 
sufficiently small 77 the OBNSLA converges quadratically. The quadratic decrease breaks down when the 
value of Pk becomes as small as an order of magnitude of rj. This result is consistent with Theorem 
@1 Figure |5(b)| depicts the interference decrease and the bound on it as given in (l34b versus the number 
of transmission cycles. It shows that the asymptotic level of the interference to the PU is bounded by 
0{if). 

The bounds in this paper are derived under the assumption that the OC holds perfectly. In practice, how- 
ever, q(n) is affected by measurement error such as noise. Furthermore, the channel matrices H pp , H sp 
vary with time, a fact that may also affect the function q(n). For example, consider a case where H pp is 
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Fig. 5. Simulation results for different values r\ of the OBNSL algorithm to obtain the null space of H with n± = 3 transmitting 
antennas and n r = 2 antennas at the PU receiver. The matrix G = H*H was normalized such that ||G|| 2 = 1. The unmarked 
lines in (a) and (b) represent the asymptotic upper bound of ( 132t and l |34t respectively on the corresponding marked line; e.g. 
the solid unmarked line is a bound on the solid line with squares. We used 200 Monte-Carlo trials where the entries of H are 
i.i.d. complex Gaussian random variables. 



a scalar and the SU extracts q(n) by listening to variations in the PU's transmit power. Assume further 
that the gain of H pp is decreased between the two consecutive transmission cycles and that the PU 
compensates for this by increasing its transmit power. In this case the SU might mistakenly deduce that 
the interference it inflicts on the PU has increased. In addition, H ps is also time-varying, which leads to 
some discrepancy between the estimated null space and the true null space. In what follows, we show 
in simulations that the derived bounds are still useful in practice if the values of r\ are not "too". A 
full theoretical convergence analysis of the O/BNSLA in practical conditions, which extends the bounds 
derived in this paper to account for measurement noise and time variations in the channel is a topic for 
future research, beyond the scope of this paper. 

Figure [6] presents simulation results for an identical scenario as in Figure [4] except for H ps which is 



generated assuming that both the SU-Tx and PU-Rx are fixed. H ps was generated according to 112 1H . 
which represents a fixed Rx-Tx channel where one antenna is 1.75 m in height and the second antenna is 
25 m in height. The Rician factor and the Doppler spread of H ps were determined according to Equations 
(13) and (14) in I21I . The result shows that for an interference reduction smaller or equal to 37 dB, the 
bound in (l34l predicts the behavior of the interference reduction (i.e., it decreases as rj 2 ) of both the 
OBNSLA and the BNSLA. Furthermore, it is shown that the BNSLA and the OBNSLA have the same 
asymptotic properties; i.e. convergence rate and asymptotic interference reduction. 

VII. Summary and Future Research 

This paper proposed a new algorithm termed the One-bit Null Space Learning Algorithm (OBNSLA), 
which enables a MIMO CR SU to learn the null space of the interference channel to the PU by observing 
a binary function that indicates the variations (increase or decrease) in the PU's SINR. Such information 
can be extracted, for instance, from the quantized version of the variation in the PU's SINR, or in the 
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Fig. 6. Interference reduction (marked lines) of the OBNSLA, as a function of Jacobi Sweeps (a), and as a function of rj (b). 
The unmarked lines represent the bound in ( 134) for the corresponding marked lines with the same pattern, e.g., in Subfigure 
(a) the dotted-dashed unmarked line represent the bound on the OBNSLA's interference reduction with r\ — —8 dB, which is 
represented by the dotted-dashed line that is marked with circles. The numbers of antennas are nt B = 2 nt p = 2, n Tp = 1 
n rs — 2. The results were averaged over 1000 Monte-Carlo trials. 



PU's modulation. We also provided a convergence analysis of the OBNSLA, which also applies to the 
Blind Null Space Learning Algorithm (BNSLA) that was recently proposed It was shown that the 
two algorithms maintain the "good" convergence properties of the Cyclic Jacobi technique, namely a 
global linear and an asymptotically quadratic convergence rate. It was also shown in simulations that 
just like in the Cyclic Jacobi technique, the OBNSLA and the BNSLA reach their quadratic convergence 
rates in only three to four cycles. In addition, we derived asymptotic bounds on the maximum level of 
interference that the SU inflicts on the PU. The derived bounds have important practical implications. 
Due to the fact that these bounds are functions of a parameter determined by the SU, it enables the SU 
to control the maximum level of interference caused to the PU. This gives the OBNSLA (or the BNSLA) 
a useful stopping criterion which guarantees the protection of the PU. The analytical convergence rates 
and interference bounds were validated by extensive simulations. 

We consider the theoretical analysis of the OBNSLA and BNSLA under measurement noise as an 
important topic for future research. Note that in the presence of noise, the analysis of the two algorithms 
is not identical since the BNSLA relies on a continuous-valued function of the PU's SINR, whereas the 
OBNSLA relies on a binary function. Noise, which is continuous-valued, will thus affect these functions 
and hence the performance and convergence of the two algorithms quite differently. 



DRAFT 



22 



Appendix A 
Proof of Theorem [3] 

Consider the first sweep of the BNSL algorithm; i.e. k = 1,2, ...,n t (nt — l)/2. Denote the number of 
rotated elements in the Ith row by b\ = n t — I and let 



£;=A = (2^-i-/)z/2; z(z, *) = ££? l[A fc ] i+M l 2 ; ^fc)^;^^^) (37) 



Note that W(0,k) = Pj*. In every sweep, each entry is eliminated once; we therefore denote A^'s p,q 
entry before its annihilation as g q , p (t) where t denotes the number of changes since k = 0. After g q)P {i) 
is annihilated once, it will be denoted by g q , p (t) where t is the number of changes after the annihilation. 
The diagonal entries of A& will be denoted by x since we are not interested in their values in the course 
of the proof. This is illustrated in the following example of a 4 x 4 matrix 



1 <7l,i (0) <7i, 2 (0) 01,3(0) 51,4(0) N 

52,1 (0) 52,2(0) <72, 3 (0) 02,4(0) 

03,1 (0) 03,2(0) 5 3,3(0) 53,4(0) 

V 04,1 (0) 04,2(0) 54, 3 (0) 54,4(0) J 
A 2 

( x 51,2(0) e 51,4(2) N 

52,i(0) x 52,3(2) 52,4(1) 

e 53,2(2) x 53,4(1) 

^ 54,i(2) 54,2(1) 54,3(1) x j 



x e 51,3(1) 5i,4(l) 

e x 52,3(1) 52,4(1) 

53,i(l) 53,2(1) x 53,4(0) 

\ 54,i(l) 54,2(1) 54,3(0) x J 
A3 

t x 01,2(1) 01,3(0) e 

02,i(l) x 5 2j3 (2) 52,4(2) 

53,i(0) 53,2(2) x 53,4(2) 

y e 04,2(2) 54,3(2) x y 



For arbitrary n t , after the first c\ sweeps A Cl 's first column is equal to the following vector: 



(38) 



[x,g2,i(n t - 3), ...,5^-1,1(0), e Cl ] 



T 



(39) 



and 



Z(l, Cl ) < |52,i(ni-3)| 2 + ... + |5„ t -i,i(0)| 2 + |e Cl | 



(40) 
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From ([TOt it follows that for q = 2, n>t, 

9q,i( n t-q-l) = cos(^ nt _i)5 9) i(n-g'-2) -e^"*- 1 ^ int (l)sin(0 nt _i) 

(41) 

&,i(l) = cos (Vi) - e^ 3 3 g , g +2(l) sin (0 3 ) 

<? ? ,i(0) = e q -!C08(e q ) -e^"g q>q+1 (l) sin (6q) 

where g qi i(— 1) = ei. The following bounds on {<? g ,i(0K!lo 9_ ^ obtained recursively (i.e., by obtaining 
a bound on g qi i(0), substituting and obtaining a bound on g qt i(l) and so on) 

g q>1 (n t -q-l)< |e,_i JX^ 1 cos - E^ 1 ^ sin ft) g qJ+1 (l) U^+i cos (<?„) | 
^Iv^fy^l+en^cos^) 

where v,y G C"*-s such that [v( 9 )]j = e^+«-^ w - +ff (l), [y(ff)]j = sin II££ ff coe 
j = 1, n t — (/, and e = max g |e 9 |. It follows that 

Ifoifa - g - 1)| 2 < |y T (g)v(g)| 2 + | e | 2 n^ 1 cos 2 (9 V ) < ||y( g )|| 2 ||v(g)|| 2 + |e| 2 fi^ 1 cos 2 (9 V ) 

(43) 

Proposition 10: 

n-l 

l|y(9)ll 2 = i-II cos ^) ^ 

i=q 

Proof: This is shown by induction. By definition 

n— 1 n— 1 

||y(g)|| 2 = ^sm 2 (^) cos 2 (^) (45) 

i=(jr U=i+1 

where 

JJ«i = l,if l>m (46) 
Assume that (l44l is true for n = m G N, then, for m + 1 (l44l and (1431 ) yields 

E™ 9 sm 2 (^) U7=i+x cos2 (^) = EST/ sm 2 (^) U7=i + i ™s*(e v ) + sin 2 (0 m ) JX1 m+ i ^ 2 (6 V ) 
= cos 2 (e m ) E™" 1 sin 2 ^) cos2 (^) + sm 2 (^ m ), 

(47) 

where the last equality is due to d46l . According to the supposition (l44l 

cos 2 (0 m ) (l - n™" 1 cos 2 (^)) + sin 2 (0 m ) = 1 - Ut q cos2 (^), (48) 



DRAFT 



which establishes the desired result. 

By substituting Proposition [10] into (l43l ) one obtains 

(nt—q \ Ti t — l 

£ |<fe +f ,(l)| 2 (l - nni.cos 2 ^)) + |e| 2 II cog2 
8=1 / t>=<? 



=Z(q, Cl ) <1 

thus, 

|5 9ll (n f - 1)| 2 < (l - n^cos 2 ^)) Z(q, Cl ) + |e| 2 



and by summing both sides of (1501) over q = 2, nt 

z(i, ci ) < z n q u (i - nri^ 2 ^)) + k - i)M 2 
< a - n-i C o +2 c°s 2 (^)) E % c i) +( n * - 1 ^\ 2 

q=2 

v v ' 

W(l, Cl ) 

< (i - n-i Co+2 co S 2 (^))^(o,o) + K - i)|e| 2 

where the last inequality is due to P Cl = W(l,c\) + Z(l,ci), W(0, 0) = Po> an d because 
monotonically decreasing sequencdj. It follows that 

Z(l, ci) = sin 2 (^ Co+2 , Cl ) W(0,0) + (nt - l)|e| 2 

where 

sin 2 (* Ci _ 1+2 , Ci ) = 1 - rc= Ci _ 1+ 2COS 2 (^) 
and 9i is an angle that satisfies \9i\ < Thus, 

P C1 = W(l, ci) + Z(l, ci) < W(0, 0) = P 

substituting (l54b we obtain 

W(l,ci) < W(0,0) cos 2 (* 2 , C1 ) - (nt - l)|e| 2 
8 Forsythe and Henrici |9j showed that the sequence Pk is a monotonically decreasing sequence. 
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Now that this relation is established, it can be applied to A Cl 's lower [n t — I) x (n t — I) block-diagonal, 
thus 

W(l, q) < W(l - 1, cos 2 (* Cl _ 1+2 , Cl ) - (nt - l)\e\ 2 (58) 



By substituting (1581 ) recursively into itself, one obtains 

I i i 

W(l,c t ) < W(0,0) H cos 2 (^_ I+ 2,cJ - e 2 ^b 3 J] cos 2 (^ c „_ 1+2 , c „) (59) 

j=l j=l v=j+l 

Thus 

Z(Ud) = sin 2 (vP C! _ I+Vl ) W(J - + (nt - /)|e 2 | < W(0,0) sin 2 (* C! _ 1+ 2 ! c ! ) 1^=1 cos 2 (*c,_ I+ 2,c,) 

- M 2 Efc\ bi nS + i cos2 (*c-i+a,c) + K - Ok 2 ] 

(60) 

After a complete sweep 

= E?=T 2 z(i,c nt - x ) + | e | 2 = E?lT 2 z(i, Cl ) < w(o,o)Er=T 2 sm 2 (* C| _ 1+2)C( ) n^cos 2 (^„ 1+2 , Cj .; 

- Er=r 2 n 2 e;=i 6j iti+i cos 2 (^_ 1+2 , c j + k 2 i ef=t V* - o 

(61) 

where the first equality is due to the fact that for k = q + 1, c nt , the sum of squares of the /th column 
remains unchanged; thus, Z(l,k) = Z(l, q),Vfc > q. Similar to proposition [TOl it can be shown that 

ELI sm 2 (t,) n -=i cos 2 fa) = 1 - UU cos 2 fa). Thus 



p 2 ^, < w(o, o) (i - np 2 cos 2 (* Ci _ 1+2)CJ )) - Er=i 2 e 2 eJ=i h cos 2 (* c „_ 1+2 , c j + 1a eli 1 ^ 

(62) 

From (1551 we have 

cfc; 

cos 2 (lV 1+2 , Ci ) > n cos 2 (^,) (63) 

and therefore 

2 ^ wen m f i _ rr™- 2 rr c ? ™„2 



Pet, < w(o, o) ^ - n -=i ixl Cj _ 1+2 COS- , 

- Er=i 2 n 2 e5=\(- - j) ntj+i n?= c „_ 1+2 cos 2 ^ + 

Recall that |0j| < 7r/4, therefore 

< W(0, 0) (1 - 2-("" 2 )(- 1 )/ 2 ) - |e 2 | (Er=i 2 Ej=i(n " j)2T-^+l+9"- 4 5 _ (nf_-) 

< W(0)(l-2-(- 2 )(- 1 )/ 2 ) + |e 2 |^^ 

(65) 
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It remains to relate e to the accuracy of the line search 77. Note that the error e in (I65T ) is due to (fT71) 
which is a result of the two finite-accuracy (of rj accuracy) line-searches in (fl2l . and ([T"3T l. If 77 were 
zero, A/j's Z, m off diagonal entry would be zero after the kth sweep, i.e. 



where 



u(el4) = (66) 

u(0,0) = |[R J)m (e^)A fc R* m (0,0)]i im | 2 = ^ 1 (e,0)+« 2 (0 ) 0), (67) 

n 1 (^<A) = 4(afj 2 S in 2 ( 7; , m + ^ 

/ / \ / \ \ 2 (60) 

1*2(0, 0) = (2 cos(20)a* m cos Za z fc m + + sin(20) (a* - < 



and (di,4>k) * s tne vame given in Theorem [T] when substituting G = A&. Recall that (OLcpi) (see ( fTTT l) 
is the non optimal value that is obtained by the two line searches, then 



m^ u {el4) (69) 

k 

The eixor u(6^, can be bounded because 0jj = Zof m , thus cf>l = —Za,i m + rj^ where |^| < 77, and 

= 4(aL) 2 sin 2 (Za? m + ${) < 4(a?jV < 2||G||r, 2 (70) 



u 2 (§lJl) = (2al m co S ( m )co S 2 (2§i) + sin(2^) (a^ - < m )) 2 (71) 

To bound vqftl, (jA), note that if = a^ m , then Ol = ^ € {0, vr/4} since the line search will not miss 
these points. Now for the case where afc 7^ a^ nm we have " k = 6 s k + rjg where 

flj^tan-^Sfc) (72) 

and 

2 l«« fc ml COS (w) 

Xfc = fe k ( 73 ) 

a rn,m a lJ 

Note that 

«2(0fc.$) = (2 cos ( V<t> ) at m (cos (20|)-2 W sin (29*)) + (a* - a* ) (sin (20*) + 2 cos (20*) r^f 
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where (9*,<ft*) is a point on the line that connects the points (0l,4>y, (O^cj)^). By substituting (l72l ) we 
obtain 

2 



2 cos(^)af m +x fc af , — z fc aj; 



4r/ e sin (29*) cos (r^) af m + 2rj e cos (20*) f af z - a; 



(74) 



Using (1731) and the fact that the sinusoidal is bounded by one, and because \rjg\ < n, it follows that 

2 



MH'H) ^ 4? ? 2 2 l sin ( 2 ^)l«/ fc m + cos ( 2 ^) 



a l,l a m,m 



< in 2 (4sin 2 (2r)|a^| 2 +2sin(4r)|af; m ||4 - a k mm \ + cos 2 (2r)|af i - a 

< An 2 (2|af im | 2 + 2sin(40*)KJ|a* - a k mm \ + |ag - a*J 2 + 2|a* m 



U20l,4>l) < W (2||G|| 2 + V2||G||||G|| + ||G|| 2 ) 



|e| 2 = max k u(§Ul) < 2(7 + 2^/2> 2 ||G| 



Thus 



This expression is substituted into (1651 ) and the desired result follows. 



(75) 



(76) 



(77) 



□ 



Appendix B 
Proof of Theorem |4] 

Without loss of generality, we assume that W(0, 0) < 5 2 /8 where W(k, I) is defined in d37H 9 |. We first 
prove the theorem assuming that G's eigenvalues are all distinct. From (l42l it follows that 

\~g qjl (n t -q-l)\ 2 < E^" 1 ^ (ty) 2 + e 2 ll^ 1 cos 2 (9 V ) (78) 

Similar to the derivation of (l50l l. but without applying Proposition [lOl one obtains 

\g qA (n t -q-l)\ 2 < Z(q, cy) Z]^ sin 2 (Oj) + |e| 2 < Cl ) E^ 1 sin 2 (9j) + |e| 2 (79) 
and by summing both sides of d79l ) (similar to the derivation of d5lT )) over q = 2, nt it follows that 

rat 

2(1, ci) < (E^ 1 sin 2 (%)) ^ Z(q, ci) +(n t - l)|e| 2 < (^=2 sm 2 (OM W(0, 0) + (n t - l)|e| 2 

0=2 f 

W(l,ci) 

Now that we have established this relation we can apply it to the reduced nt — I + 1 lower block 
9 I.e. let fco be the smallest integer such that Pk < 8 2 /8 and (Zfc , "ife ) = (1; 2), we set Afe = G). 
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diagonal matrix and obtain Z(l,q) < (j2f =Cl l+1 sin 2 (9jfj W(0, 0) + (n t - l)\e\ 2 . After a complete 
sweep we have 



< Er=i" 2 ^c„ t -i) + | e i 2 = Er=i" 2 ^Q)+ki 2 

n t (n t -l)/2 . 2/q \ , |,2|V^n t -l/ 



< w(o, o) Epr ;/ sm 2 (^) + |6 2 i Er=rK - o 

We now relate ^K" 1 )/ 2 s in 2 (0,) to jy(0,0) ( recall that P 2 = W(0,0)). Note that |af, ~ fc 12 



| af t - A; - 4m + A m + A; - A m | 2 > | - A m | 2 - | af; - A; | 2 - | 4m - A m | 2 , furthermore, by Q12 
1], there exists a permutation to {Aj}" = i 1 such that 



Theorem 



14 -^1 <\/2P fe , (81) 

thus, 

14 - Ai| < 5/2, (82) 

and 

14 -4ml > 25 -5/2 -5/2 = 5- (83) 

Recall that the optimal rotation angle satisfies tan(2#^) = 2|4 m 1/14 z ~~ a m k m k \ while the actual the 
rotation angel is 

el = ei + Ve (84) 

It follows that 



sin 2 (^)| < | sin 2 (^)| + \rieMMl)\ < l|2^| 2 + M tan(20;J)| < ^ tan 2 (2^) + \ Ve \ tan(20j>)| 

(85) 



- <5 2 ~t~ z I'/<?I S — 5 2 8 



Therefore 



(86) 

By substituting (l86l ) into (l80l ) one obtains 

< ^(0,0) (jsW(0,0) + W-y>l»l v^fcj) + ? (n? - nt) , (87) 

It remains to relate Hg in d84l) to the accuracy of the line search rj. Recall that the calculation of 6i relies 
on the calculation of (f>i. Thus, as a result of the finite accuracy of the line searches, rjg depends on 77^ as 
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well, as we now show. Form the proof of |4l Theorem 2] we know that if an accurate line search were 
invoked, it would produce 4>j. = —Zaf m . However, the actual line search yields 4> k = —Za^ + r]^, where 
\v</>\ < V- Thus, 9 k is obtained by searching the minimum of a perturbed version of S(A k , r^ m (9, 
i.e. 

S(A k ,r l>m (e, 4> J k )) = h k (cos 2 (6)al l - cos^) sin(20)af m + sm 2 (9)a^ m ) (88) 

We first assume that afc ^ a<mm- From the proof of |4l Theorem 2] , the optimal value of 9 is Ol = 
h tan -1 (p k ) , where x k = ir~r4 ■ If one takes into consideration the non-optimality of the line-search 
which obtains 4>\ and ignores the non-optimality of the line search that obtains 9 k , then the minimizer 
of (l88l) would be 9 s k = \ tan -1 (x k cos^)) and the difference \9 k — 9 s k \ is 

\9i - V h \ = |§tan-i (x fe co S ( W )) - ^ (x fc )| < ^gg^ < (89 ) 

where fo* | < r/ . It can be easily shown that cos 2^ xi+1 < |cob(^)| ' and because H = e l + e k- e l + W 
and |r/^| < ?], the accumulated effect of the finite accuracy of both line searches is bounded by -qg < 
rj + icos^i ■ Assuming that r/ is sufficiently small, (e.g. rj < 7r/20) we obtain 

m < 6r7/5 (90) 
By substituting d90]) and C3) into ([87]) it follows that 



Pl t _ 1 <W{^Q,)(j ! W{^Q,) + r ] ^^ (91) 

Thus, as long as w is negligible with respect to W(0, 0), the BNSLA will have a quadratic convergence 
rate for G that does not have multiple eigenvalues; i.e all eigenvalues are distinct. This is not sufficient 
since we are interested in a matrix G nt — n r with zero eigenvalues. 

To extend the proof to the case where the matrix G has n t — n r zero eigenvalues and n r distinct 
eigenvalues we use the following theorem: 

Theorem 11 (hldl Theorem 9.5.1): Let A be an n t x nt Hermitian matrix with eigenvalues {A/}™^ 
that satisfy 

Ai 7^ A2 7^ • • • 7^ A„ r 7^ A„ r+ i = \ Ur +2 = • • • = A„ t = A (92) 
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Consider the following partition: 



Ai B 
B A 2 



(93) 



where Ai is n r x n r and A 2 is (n t — n r ) x (n t — n r ) and let 5' > 0. If ||(Ai — AI) x || < 1/5', then 



AI|| < ||B|| 2 /5' 



(94) 



To apply Theorem [TT] to the modified O/BNSLA, we need to show that A& satisfies its conditions. 
This however is only satisfied in the next Jacobi sweep; i.e. Theorem [TT] can be applied to A& with 
k > m + 1. To show this, note that (l82l and (l83l are satisfied by A^, k < m for some permutations of 
the eigenvalues. Thus, due to the permutation in (|22~1 ). A&, k > m satisfies (f82l ) and (f83l) . for the ordering 
of d92l ). For the rest of the proof, it is assumed that k > m. Let Aj, A§,B* be A^'s submatrices that 
correspond to the partition in (|93l . Recall that in our case, A = 0, thus, (|82l ) implies that 



|Af|| 



and also implies that > 55/2, V0 < I < n r . Furthermore, by [119 



Corollary 6.3.4] 



|Ai(Af; 



41 < l|Afc||off < S/2 



(95) 



(96) 



Thus 



Ai(Af) > 0, V0 < I < n r 



(97) 



and therefore, the matrix A^ is invertible, and from (|93T l, it follows that ||(A^) 1 || < 1/5, which enables 
us to apply Theorem [TT] to obtain 



|A||| < \\B k \\ 2 /5 



(98) 



To show that d98l ) leads to quadratic convergence, one must show that the affiliation of the diagonal 
entries in the upper n r x n r -block of A^ remains unchanged and that the eigenvalue that correspond 
are arranged in decreasing order, i.e. 



I = arg min | A/ 

l<m<n t 



arg min |A{ — o^|, VZ G {1, ...,n r } 

±<m<n t 



(99) 



and 



A/ > A m , V/ < 771 



(100) 
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2 

a lk,h ~ a m k ,m k 



To show (HU), note that 

< sin 2 (9 k ) (2 cos(e k )al >mk cos (fo - Za£ iTOk ) + sin (0 fc ) (a£ A - 

and that for every 6 k such that l k < n r , (l85l ) is satisfied. Thus 

< Bin 2 (<fc) (< mfc + ((< mfe ) V + 2^ ^ 

< sin 3 (fc) (at mk + ((<„„) 2 + 2%< mfe ) 1/2 ) 2 < sin 2 (0 fe ) (a* >|ftt + (£ 2 / 4 + 2%«5/2) 



< sin 2 (^: 



'*) { a tm k + « (1/4 + m) 1/2 ) 2 < t (1 + (1/2 + 71/2+^ 

By restricting t] < 1/100 and considering (1901 it follows that 

Kz, - < 0^ (103) 

which establishes d99l ). 

Now that d99l ) is established, (1 1001 ) immediately follows and for every I, m such that I ^ m and 
1 < I < n r , we have \af t — a^ml — And ( [80b can be written as 



1/2 1 

(102) 



jv+m< Er=7 1 ^^„ r +m) + i6 2 iEr=iK-o 

< W(0, m) E -L"! sin 2 (%) + E?=;!+i ^ c„„ + m) + |e 2 | EF=i(^ - 

Recall that |e| 2 < maxku{9 k , (j> k ) where ui(9 k , <j) k ) an d U2(9k,4>k) are defined in d70l and (TTTb . From (l70l 
and ([75) u(0 k ,<j) k ) < 4r/ 2 (5P fe 2 + 4P fe ||G|| + ||G|| 2 ). Because \af hh -<4 fem J > 5 for m < k < m + c nr , 
is satisfied and similar to (|9ll we obtain 



W(0, m) (^(0, m) + r,M0fp)) 
+2 (2n t n r - n 2 - n r ) r/ 2 (5W(0, m) + 4^(0, m)||G|| + ||G|| 2 ) + EZnl+i Z (h Cn r + m) 



It remains to bound the term Ej^c^ +1 Note that for every 9 k such that m < k < c„ r +m, 

(I85T ) is satisfied. Let Q = {(/, m) : 1 < Z < n r < m < nt}. Note that for every k such that (Z&, m^) € Q, 
and a^m* are located in and A2 respectively. Thus, 

laJ+JJ 2 < |< m J 2 + sin 2 (0 fc )K i9 | 2 , for n - < 1 < m k (1Q6) 
|Om"^| 2 < l«m fcl9 | 2 + sm 2 (^)|af fc 9 | 2 , for m k < q < n t 
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and from 



\<] q \ 2 < \<J 2 + + 2%%^) Kj* 



(107) 



These can be bounded by 

l«K ff | 2 . |o{+£j a < W 2 (0,m) (l + + ^ 3 / 2 (0,m) (108) 
Thus, for every k G {m, m + c nr }, 

Eri +1 ^^ + -)= E 1 E l< f l 2 <of(^^) 2 ) + o(( ^ 3/ y + -) )) (io9) 

This, together with (11051 ) and (|90]> show 

< ^H^0±H) 2 ^) + O + ^ ^j0^m) jj + 2 (n 2 _ ^ ^^2 

(110) 

Since is a decreasing sequence, the desired result follows. □ 

Appendix C 
Proof of Theorem [5] 

We first prove the theorem for the case where the non-clustered eigenvalues are the largest; i.e., Aj > 
Aj + i + (5 c and Aj — A > S c for i = 1, ...,n r — v. Note that Aj = A+fi_ nr _ t , for i G L2 = {re r — ...,n r } 
and Aj = for i = n r + 1, i%t- Without loss of generality, we assume that W(0,0) < 5 2 /8 where 
W(k,l) is defined in ([37]). Let V fc AV£ = A k be A fc 's EVD, and let A k = V fc AV*, A k = V fc AV* 
where 

A = diag(Ai, • • • ,X nr ,- v ,^-- 0_^_0 ) 

t) n t -v-n r (111) 

A = diag(0_^0,6,--- 0_^0 ,) 

n r —v n t —n T —v 

Let L\ = {1, n r — v}, L3 = L\{L\\J L2) and L s = (Li x L)U (L2 x L3), L c = L s n{(Z, m) : / < m}. 
By combining ([8D and the condition P% < 5%/8, it follows that ([82]) and ([83]) hold for 5 = 5 C . Thus, 
due to the permutation in ([221 . the inequalities (l82l and (l83l are satisfied for A^, k > m, V(Z,m) G L c 
and 5 = 5 C . In the rest of the proof, we assume that k > m. Because \a k t — a^ m | < 5 c ,V(/,m) G L c , 
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Afc can be partitioned as 



Afi A-12 Af 3 

\k \k \k 

A 21 A 22 A 23 



A k 

A 31 



\k A k 

A 32 A 33 



(112) 



where A^ 2 G «)x(n r -?;) an( j ^fc^ £ C" X1 '. The idea behind this partition is that the diagonal entries 
of Aj^ are separated by more than S c , and two diagonal entries such that each belongs to a different 
diagonal block (i.e. An, A22, A33) are also separated by more than S c . Now it is possible to use 1114 . 
Lemma 2.3] which asserts that 

P? 

2,3. (113) 



p2 

lAgHoff < ^,for I 



where ||A?5|| g is the sum of squares of A^j's off diagonal entries. 



To show that (II 131 ) establishes d29l ) we first show that the affiliation of the diagonal entries in the 
upper A^ -block remains unchanged and that no diagonal entry leaves the A\ 2 an d A33 blocks. To be 
precise, for i = 1, 2, 3 



Rk+i{v) G Li if v G Lj,and Rk(v) = v if v E Li, 



(114) 



where 



Rk(v) — argmin|A„ 
2eL 



41 



This follows from (1 1 1 b and because for every k such that G L c , 

replacing 5 by <5 C . Thus, similar to (1 1 02b . for every k such that (lk,mk) G £ s 



(115) 

is satisfied with 

2 



a 



fe+i 



< 



1 + 4770) (l/2 + a/1/2 + r/e) . By taking 77 < 1/100 and considering (|90]> it follows that 



a f Z I < 0.655 c , which establishes (II 151 ): and therefore, for every (l,m) G |a* 
Similar to the derivation of (I105l i. 



><5 C - 



Pr_. .. . r +m < W(0, m) ( £W(0, m) + r/^ c ^/^(0, fc) 



+ ic nt . v . rV 2 (5W(0, m) + 4^(0, m)||G|| + ||G|| 2 ) + E&«+r+i c„,__ r + m) 
and similar to the derivation of (111 Ob . we obtain 



(116) 



< n | ( W(0,m) 
n - U \ \ S c 



+ 



rjW 3 / 2 {0,m) 



+ 



+ 2{n 2 t -n t ) r/ 2 ||G| 



Since Pf. is a decreasing sequence, the desired result follows. 



□ 
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Appendix D 
Proof of Corollary [7] 

The corollary follows from Theorem |3] and from the following proposition: 

Proposition 12: Let b > 0,0 < p < 1 and let a n be a non-negative sequence that satisfies 

a n +i < pa n + b, Vn G N, (117) 

then, 

limsupa n < (118) 

n 1 — p 

Proof: We first assume that for some n G N a n > j^-. In this case we have a n+ \ < a n which 
means that a n is a monotonic decreasing sequence as long as a n > j^—. In the case where a n < 
we have a n+ \ < y^-. These mean that either a n converges to a limit £ > or that it satisfies 

(II 18b . Assume that the previous statement is true, then for every e > 0, there exists n e G N such that 
£ — e < a n < £ + e, Vn > n £ . By substituting it into (II 171) . i.e., substituting £ — e for a n+ \ and £ + e for 
a n it follows that for every e > 0, < 6 + e(l + p). This is equivalent to £ < ^ + e ^+^ , Ve > 

which is a contradiction. □ 
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